Deterministic evolutionary game dynamics in finite populations 
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Evolutionary game dynamics describes the spreading of successful strategies in a population of 
reproducing individuals. Typically, the microscopic definition of strategy spreading is stochastic, 
such that the dynamics becomes deterministic only in infinitely large populations. Here, we intro- 
duce a new microscopic birth-death process that has a fully deterministic strong selection limit in 
well-mixed populations of any size. Additionally, under weak selection, from this new process the 
frequency dependent Moran process is recovered. This makes it a natural extension of the usual 
evolutionary dynamics under weak selection. We analytically find simple expressions for the fix- 
ation probabilities and average fixation times of the new process in evolutionary games with two 
players and two strategies. For cyclic games with two players and three strategies, we show that the 
resulting deterministic dynamics crucially depends on the initial condition in a non-trivial way. 



I. INTRODUCTION 

Evolutionary game dynamics results from the transfer 
of economic ideas to biology [IHl]- In economics, ra- 
tional players try to find the best strategy to maximize 
their payoffs. In biology, those individuals who use the 
best strategy obtain the highest reproductive fitness and 
spread in the population. 

Traditionally, evolutionary game dynamics is consid- 
ered in infinitely large, well-mixed populations. This 
typically leads to the replicator dynamics, a system of 
nonlinear differential equations governing the evolution- 
ary dynamics [5-8 . For any composition of the popu- 
lation, the replicator dynamics determines deterministi- 
cally the direction and velocity of evolutionary dynamics. 
The replicator dynamics can be derived from microscopic 
models of strategy spreading, which are typically stochas- 
tic |9HT3] . The precise definition of strategy spreading be- 
tween individuals can have decisive consequences for the 
dynamics, in particular in structured populations [14]- 

EI]. 

Since microscopic models of strategy spreading are typ- 
ically stochastic, evolutionary game dynamics in finite 
populations can only be characterized in a probabilistic 
way. The most important quantities are the probability 
that a mutant takes over a population and the average 
time for this process [22H25] . Different models for strat- 
egy spreading have been proposed. A popular model is 
to choose two players, Harry and Sally, at random and to 
let Harry adopt the strategy of Sally with a probability 
given by the Fermi function, (1 + exp[— /3(7r^ — 7r^)])~^, 
where tt^ is the payoff of Harry and tt^ is the payoff of 
Sally [26H29] . The parameter P measures the intensity 
of selection. For /3 <C 1, selection is weak and strategy 
spreading is essentially random. For /3 1, selection is 
strong and only strategies that are more successful will 
be imitated. For /3 ^ oo, the direction of the process for 
two strategies becomes deterministic and thus the fixa- 
tion probability is either or 1. However, even in this 
case, the process is only semi-deterministic, as the time 
of fixation remains stochastic 29 . 



Here, we introduce a variant of the Moran process 
which leads to a fully deterministic evolutionary process 
in finite populations under strong selection. For weak 
selection, we essentially recover the transition probabili- 
ties of the standard frequency dependent Moran process 
under weak selection. 

We describe evolutionary game dynamics in symmetric 
2x2 games defined by the general payoff matrix 
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An A player will obtain a when playing against another 
A or b when playing against B. Choosing strategy B 
results in either obtaining c (against A) or d (against B). 

The average payoffs are obtained from pairwise inter- 
actions with all other individuals in the population of 
size N. This is the standard assumption and refers to 
the fact that the population is well-mixed, i. e. there is 
no explicit population structure. Excluding self interac- 
tions, this leads to 
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where i is the current number of A players in the pop- 
ulation. Individuals with higher average payoffs pro- 
duce offspring (or are imitated) with a higher probability. 
Thus, reproductive success is based on the payoff from 
the game. The intensity of selection P controls the im- 
portance of success in the game for reproductive success. 
The larger the intensity of selection, the stronger the in- 
fluence of the average payoff difference on reproductive 
fitness. 

The paper is organized in the following way. In Sec- 
tion |n| we introduce the birth-death process as a general 
framework of evolutionary dynamics between two types 
in finite, well-mixed populations. In particular, we ad- 



dress the probability and times of absorption. In Sec- 
tion |III| we give three exphcit analytical forms for the 
microscopic dynamics and we discuss the possibility to 
analyze strong selection in each case. We show that the 
standard Moran process and a previous generalization 
do not allow a fully deterministic strong selection limit 
and propose a new generalization of the Moran process 
with selection at birth as well as selection at death. In 
Section [TV| we perform the strong selection limit analyt- 
ically for the new process. In Section [V| we consider the 
process with selection at birth and death for two player 
games with three strategies, namely the the Rock-Paper- 
Scissors game. Finally, in Section [VT] we conclude and 
discuss our findings. 



II. EVOLUTIONARY GAME DYNAMICS IN 
FINITE POPULATIONS 

In this section, we recall some important properties of 
stochastic evolutionary game dynamics in finite popula- 
tions. For simplicity, we restrict ourselves to birth-death 
processes in which the number of A players can change 
at most by ±1 in each update step. 

Let i be the number of A individuals players in a pop- 
ulation of size TV > 2. The number of B players is given 
by N — i. The transition probabilities to move from i 
to z + 1 and to i — 1 are denoted by T^^{/3) and T-{/3), 
respectively. The probability to stay in the current state 
is thus 1 — T^{f^) — T-~(/3). These microscopic details do 
not have to be specified further at this point. The only 
requirement is that the expressions for T^{/3) ^ are 
analytic in system size, payoffs and intensity of selection. 
It is also assumed that the population size remains con- 
stant. Mutations are excluded, such that a strategy that 
is lost will not re-appear in the system. 

In the continuous limit N ^ oo, the state of the sys- 
tem X = i/N becomes a continuous variable, strategy A 
can have any abundance and we recover a deterministic 
differential equation j9UT3]. This allows to compute the 
fixed points of the system. There are always fixed points 
at X = and x = 1. In addition, there can be a third 
fixed point at = {d — b)/{a — b — c -\- d)^ which is un- 
stable when a > c and b < d and stable when a < c and 
b>d. 

In finite population models, stochasticity does not al- 
low the definition of fixed points. However, the bound- 
aries i = and i = N are absorbing due to the ab- 
sence of mutations, Tq^(/3) = and T^{P) = 0. For 
recurrent Markov chains {T^{/3) > for < i < N)^ 
the system will eventually be absorbed at the bound- 
aries. The probability that a given number of i 
A players will reach the absorbing boundary at N is an 
important quantity to describe the process. In addition 
to this fixation probability, the unconditional and con- 
ditional fixation times, ti{/3) and tf{/3), characterize the 
stochastic process p4l |3QH32| . These two average times 
are the expectation values of the number of time steps it 



needs either to reach any homogenous state (all A or all 
B) or to reach fixation at all A under the condition that 
this event occurs. In the following we recall recursions 
for these three quantities which can be solved regardless 
of the details of the birth-death process. Each solution 
is only based on the microscopic transition probabilities 



A. Fixation probability 

The probability of fixation describes the prob- 

ability that a given number i of new A mutants in a 
population of B will reach fixation at all A. Since the 
homogenous states are absorbing;, we have = as 

well as = 1. For all the intermediate states we can 

write a balance equation for the probability to fixate at 
ah A{i = N), 



With the boundary conditions (/>o^(/3) 
this can be solved recursively [4, 24 . We obtain 
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The probability to fixate at the pure state all B starting 
from i A individuals is given by = 1 — 



B. Unconditional average fixation time 

The average time (measured in elementary time steps) 
it needs to reach fixation at one of the homogenous states 
(^i = OY i = N) starting with i players of type A is 
denoted by Obviously, to(/3) = and tAr(/3) = 

0. The unconditional fixation times also fulfill a balance 
equation with the transition probabilities describing the 
rate of change, 

+ (i-7;+(/3)-7;-(/3)) uip) 



(6) 

This is a recursion equation for the unconditional mean 
exit times or average times of fixation. Its solution for 
the unconditional average fixation time reads [32l [33] 
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Next, we address the time it takes to reach a particular 
absorbing state. 

C. Conditional average fixation time 



have /^(^) = 1 -/? + /? TT,^ and /f = 1 - ^ + /5 vrf . 
The quantity < /3 < /3max determines the intensity of 
selection. The transition probabilities of the Moran pro- 
cess are thus given by 



Under the condition that the process reaches the ab- 
sorbing state at all A, z = A/", the average time of fixation 
starting from z, is tf{l3). Following [24 , we start from 
the recursion 

0f (/?) = T-{p)<ktimttm + ^) 

+ {l-T+{P)-T-{P)) </.f(/?)(if(/?) + l) 

+T+{fi)<ktimttm + '^)- (9) 

The conditional average fixation time is conditioned upon 
fixation at all A, which occurs with probability 
Thus, the product of probability and conditional time of 
fixation appears in the recursion. There can be a finite 
expectation value even for vanishing fixation probability, 
because (l)f{p)tf{P) -> does not imply tf{P) 0. For 
the average fixation time under the condition of absorp- 
tion at ah A we find El Ea [31 [35], 



N-l k 



iP) TT 



^ti^mY^iim (10) 



N-l k 



4>m 
(/?) 



k=i m- 



(11) 



There is an analogous expression for the fixation time 
under the condition that strategy B gets fixed in the 
population, tf{/3), [32 . 



As the expressions ([5|, ([7| and (10) are functions of 
the T^{/3), the study of a strong selection limit has to 
be performed in the transition probabilities. In the next 
section, we introduce a process with the analytical strong 
selection limit {f3 ^ oo) ^ 1 and T~ {f3 ^ oo) ^ 
(or the other way around) , and we show that the resulting 
dynamics is fully deterministic in this limit. 



III. THE MORAN PROCESS 

A. Selection at birth and random death 

A standard model for evolutionary dynamics in finite 
populations is the frequency dependent Moran process 
[22]. This process incorporates the following steps: An 
individual is selected at random, but proportional to 
its fitness. This individual produces identical offspring. 
The offspring replaces an individual randomly selected 
for death. Fitness / is a convex combination of the av- 
erage payoffs from the game, irf and tt^, and a back- 
ground fitness, which is usually set to one. Thus, we 
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Selecting proportional to fitness implies that fitness is 
positive. Thus, for payoff matrices with negative entries, 
the intensity of selection P cannot exceed a threshold 
/^max- This process does not have a generic determinis- 
tic limit with arbitrarily strong selection intensity and 
remains stochastic with random death. 

A possibility to extend the Moran process to higher 
intensities of selection is to choose fitness as an exponen- 
tial function of the payoff, i.e. ff^{P) = exp{-\-f3 irf) and 
/f = exp(+/37rf ) [361 [37]. Now, the intensity of se- 
lection /3 can be any positive number. For /3 -> oo, the 
fitter individual is al ways selected for reproduction, com- 
pare equations (12a) and ( |12b[ ). The direction of the pro- 
cess becomes deterministic. But due to random death, 
the system can remain longer or shorter in a particu- 
lar state. Thus, the process remains stochastic in what 
concerns the times to fixation. With two strategies, we 
have a semi deterministic process with deterministic di- 
rection and stochastic speed [29 . If there are more than 
two strategies, random death can also change the com- 
position of the less fit types in the population. This can 
affect the direction of selection, as the fittest type can 
change due to frequency dependent selection. 



B. Selection at birth and death 



Here, we introduce a birth-death process that recov- 
ers the usual results for weak selection, but also leads to 
fully deterministic asymptotic behavior for strong selec- 
tion. The process has deterministic microscopic dynam- 
ics a the T^^{/3) are zero or one. As in the standard Moran 
process, we assume that selection at birth is proportional 
to fitness. In addition to producing less offspring, individ- 
uals with a lower fitness now have a higher probability to 
die. A simple way to incorporate this is to select at death 
proportional to inverse fitness. To ensure that fitness is a 
positive number, we follow the approach discussed above 
and define fitness as an exponential function of the pay- 
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off. This leads to the transition probabihties 
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Selection at death 



is the intensity of selection at birth and is the in- 
tensity of selection at death. For /^^^ = 0, we recover the 
process discussed in Section |III| It is known that under 
weak selection many birth-death processes have the same 
general properties [23^9I^- Especially, for ^^^^^ <C 1 
the behavior of the Moran process is recovered [29l [30] . 

For simplicity, we assume /3 = /^^ = in the following. 
The transition probabilities can be written as 

i N — i 

THP)= . , ... .^.-.A....-.A.. , (14a) 
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Thus, as far as the payoffs are concerned, the transition 
probabilities only depend on the difference between the 
frequency dependent average payoffs, Att^ = irf — irf. 

The case of Att^ = is a form of neutral selection. 
In this case, the transition probabilities are T^{(3) = 
(TV — i)/N'^^ for arbitrary p. Note that for neutral se- 
lection, moving into one direction is equally probable as 
moving into the other T^{0) = Tr{0). But the prob- 
ability to leave a given interior state changes with z, 
7;±(0)^T±(0)fori^j. 

For arbitrary /3, the ratio of the transition probabili- 
ties reduces to an exponential function of the payoff dif- 
ference, 



^ g-2/3A7r, 



(15) 



Hence, the fixation probabilities of the process can be 
approximated with the closed expressions derived in [28j , 
after rescaling the intensity of selection by a factor of 
2. From this, it is clear that the usual weak selection 
behavior is recovered. 



IV. STRONG SELECTION 

For strong selection, /3 ^ oo, the asymptotic behavior 
of the transition probabilities depends only on the sign 



of the payoff difference. We focus on the generic cases 
AiTi ^ to discuss this li mit. The limiting cases can be 
obtained from equations (14a) and (14b) and yield 
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for Att,- < 



as well as 



1 for AiTi > 



1 for Att,- < 



(16) 



hm Tr{p) = ^ . (17) 

I for Att. > 



Since \im(3^oo{T,^ (P) +^"(/^)) = 1, the probability to 
stay in the state i {0 < i < N) vanishes for /3 ^ oo and 
nontrivial payoff difference, Att^ ^ 0. Thus, we have a 
fully deterministic process for arbitrary population size. 
With this, we consider the fixation probability (t)f{P) and 
average fixation times ti{P) and tf{P) in the limiting case 
of strong selection. In the following let (j)f{oo) as well as 
ti{oo) and tf{oo) denote the finite asymptotic (/3 oo) 
values of the fixation probability and times. We identify 
them in terms of the initial frequency z, depending on 
the average payoff difference for the process introduced 
above under strong selection. 



A. Fixation Probability 

Starting with equation (|4| and inserting the limiting 

cases of T^{j3) leads to 



(18) 



(/.f(^)= lim T-{I3)\4>U{^) 
+ f lim T+{/3)) 0f+i(oo) 



/3^oo 



That is, in the strong selection limit we have a very simple 
recursion for the asymptotic value of the fixation prob- 
ability, depending on the sign of the payoff difference 
Att^. When strategy A dominates strategy B {a > c and 
6 > d), we have Att^ > 0. For < i < this yields 
lim/3^oo^"(/3) = 0, and lim^^oo ^+ (Z^) = 1, which re- 
sults in ^-"^(oo) = 1 — (5o,i. In other words, the probability 
to reach the state with all B individuals is zero, except 
if there are no A individuals initially. Equivalently, for 
dominance of strategy B we obtain (oo) = 6^,1- More 
interesting cases are found when Att^ changes its sign, 
which occurs at = {N{d — b) -\- a — d)/{a — b — c d) 
when a > c and d > b or when a < c and d < b. These 
are two important classes of games: coordination games 
and coexistence games. 

Let us first focus on coordination games (a > c and 
d > b). In these games, the threshold value z* cannot 
be crossed for infinitely large systems with determinis- 
tic dynamics [40]. For finite systems under strong selec- 
tion, we observe something similar. The payoff difference 
Att^ changes sign from negative to positive and selection 
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points always away from i* toward the boundaries. De- 
pending on the initial condition i ^ i* , the fixation prob- 
ability (/)^(oo) is one or zero. 



(i) If i < 



i*, AiTi is negative and with Eqs. (16) 
and (17) we have ^^(oo) = ^^-,^(00). We start 



the recursion for the fixation probabilities with 
0f (00) = ^^(00) = 0. This yields 0f<^*(oo) = 0. 

(ii) For z > z*, Att^ is positive and Eqs. ([l6| and 
(17) yield the recursion 0f (00) = (^f_^i{(X))~StdiTt- 



ing with the maximal i we obtain 
(j)^{oo) = 1 and thus (/)^^*(oo) = 1. 



4-iM 



(iii) If z* happens to be an integer value and the system 
starts there, the first step has equal probabilities, 
7;t(/3) =T-{p) = ^. This leads to (f)ft{oo) 



1 

2 • 



In summary, for the fixation probability we find 

'0 for z < r 



for i i* 
for i > r 



(19) 



This is clearly what is to be expected because A is se- 
lected for i > i* and B is selected for i < , FIG. [l] 
Dominance of strategy A can be seen as a special case of 
coexistence with < 0. 



aries toward i*. For strong selection, the system gets 
trapped and fixation never occurs. 

(i) If i* is an integer, the system switches from i* to 

± 1 with equal probability. From i* ± 1, it always 
returns to z*. 

(ii) If i* is not an integer, we observe deterministic 
fiipping between the two neighboring states states 
ii < r, 12 > i*. 

Since fixation never occurs in coexistence games, it does 
not make sense to compute the asymptotic value of the 
fixation probability. Formally, the probability to get ab- 
sorbed in all A converges to 1 if z* > N/2 and to 
otherwise. However, it turns out that the fixation times 
diverge. 



B. Unconditional average fixation time 

The average time it takes for i A players to either be- 
come extinct or take over the population, can be 
found by solving equation ([6| recursively. To examine 
the limit of strong selection, we perform the limit on 
both sides of the balance equation, assuming that there 
exists an asymptotic value ti{oo) of the unconditional fix- 
ation time. With the previous analysis of the transition 
probabilities, this leads to 
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FIG. 1: The fixation probability as a function of selection 
intensity for a coordination game (a > c, 6 < d), give n by 
Eq. ([5|, with selection at birth and death, Eqs. (14a) and 



(|14b|. From the payoff matrix given in the figure, we obtain 
i* = M ^ IT^' which gives ~ 5.32 for our numerical exam- 
ple with N — 10. With increasing selection intensity /3, for 
any I < i* we have (l)f'{P) (straight lines), whereas for 
any m > i* we have 1 (dotted lines). The (j)t{P) 

for each i can be identified via its neutral value (/)^^(0) = 



Next, we consider coexistence games with a < c and 
d < b. In this case, Att^ changes sign from positive to neg- 
ative and selection points always away from the bound- 



t,(oo) = 1+ lim T-{p) ti_i(oo) 



lim T+iP) (20) 



This strong selection recursion has to be analyzed for 
the two different cases of behavior at the threshold z*, 
coordination and coexistence. Again, we first examine 
the coordination game {a > c and d > b). 

(i) For i < i"" ^ the payoff difference is negative, Att^ < 
0. The recursion amounts to ti {00) = 1 +ti_i(oo), 
because T+(i) whereas T' {(3) 1. With 
to (00) = at the boundary we have ti(oo) = 1, 
^2(00) = 1 + ti(oo) = 2, and eventually t^(oo) = i. 

(ii) For i > i* the payoff difference is Att^ > 0. The 
recursion from ( [2Q| ) is t^(oo) = 1 +^^+1(00). The 
transition probabilities T^{i) behave exactly in the 
opposite way as before. The time starting from 
next to the absorbing boundary is t at- 1(00) = 1 + 
In {00) = 1. Hence, we have tN-k{^) = or 
with k = N — the uncondtional averag time is 
ti{oo) = N -i. 

(iii) If the threshold is an integer and the system is ini- 
tiated there the number of steps needed to fixate 
is i* or N — i* with equal probability. Thus, we 
can compute the average time with the previous 
findings, (00) = l/2(r -f AT - T) = N/2. 
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In summary, depending on the the starting point i*, the 
asymptotic value for the unconditional average fixation 
time in a coordination game is 



ti{oo) 



2 

N ■ 



for i < r, 
for i = r, 
for i > i*. 



(21) 



In the strong selection limit t^(oo) converges to the dis- 
tance between initial state and the final state, as expected 
from deterministic motion. 

We can also infer the dynamics for games in which one 
strategy dominates. When strategy A dominates, we can 
formally set z* < and obtain ti{oo) = N -i, cf. FIG.|2j 
When strategy B dominates, the equivalent procedure 
yields ti(oo) = i. 

In case of a coexistence game, the system gets trapped 
around and cannot reach the absorbing boundaries. 
As expected, the recursions lead to t^(oo) oo. 
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FIG. 2: Probability distribution of the unconditional fixation 
time (measured in elementary time steps) of a single A player 
in a population of N —1 dominated B players. For P — 1 and 
/3 = 2, the distribution has two peaks corresponding to the 
two absorbing boundaries. For stronger selection (inset), the 
probability that the advantageous A individual goes extinct 
becomes small and fixation takes at least N — 1 time steps. In 
this case, the distribution becomes single peaked. For /3 ^ oo, 
the distribution converges to a delta peak at ti (oo) = — 1 
(payoff matrix a — 2.2, 6 = 1.5, c = 2, = 0.5, population size 
iV = 100, histograms obtained from 10^ realizations. Lines 
are guides to the eye). 



consider the fixation probability and the conditional fixa- 
tion time in a combined way. Introducing the asymptotic 
value ^^(oo) = (/)^(oo) t^(oo), the recursion |9| yields 



lim 7;-(/3) (eiM + ^f-iM) 



lim 7;+(/3) (^tiM + C+iM)- (22) 

p^oo ' 



The formulation of a similar equation for Of {oo) = 
(l — (j)f{oo)) if {oo) is straightforward. Both are ana- 
lyzed regarding the different behavior at either side of 
the threshold i*. 

For the coordination game the system reaches the ab- 
sorbing boundaries after a finite time. 

(i) If z > i*, the system fixates dX i = N with proba- 
bility (oo) = 1. Thus, 6>f (oo) = tf{oo) and we 
recover the same recursion as for the unconditional 
fixation time. This yields tf{oo) = ti{oo) = N — 
see FIG. El 

(ii) If i < i*, the system fixates at z = 0, (j)f{oo) = 0. 
Thus, we cannot formulate a meaningful recursion 
for tf{oo). In this case we observe T-~(/3) 1, 
T^{P) and we can only make a statement for 
tf{oo), which results in tf{P) = i. 

(iii) If i = i* is an integer, the system is not fully de- 
terministic as fixation of A and fixation of B are 
observed with equal probability ^. In this case, we 
obtain tf.{oo) = N - i* and tf{oo) = i* . 

In a regime where A always performs better than B the 
unconditional fixation time tf{oo) is equal to the condi- 
tional fixation time ti(oo), see FIG.js] Equivalently, when 
B always performs better, we have if {oo) — t^(oo). 

For a coexistence game, the system does not reach any 
of the boundaries but is always dragged toward i*, as 
discussed before. The recursion (22) for tf(oo) or its 
equivalent for tf{oo) are not meaningful here because 
they contain the fixation probabilities. However, in this 
case all fixation times diverge with (3. 

In this section, we have derived asymptotic values for 
the birth-death process with selection at birth and selec- 
tion at death. We have identified the underlying games 
that lead to a fully deterministic process in the limit of 
strong selection. As we have seen, the difference of the 
average payoffs Att^ as a function of the the relative abun- 
dance of type A plays an important role. 



GAMES WITH THREE STRATEGIES 



C. Conditional average fixation time 

If i A players take over the population, the asymp- 
totic fixation time under this condition, t^(oo), can be 
obtained by solving the balance equation ^ recursively. 
However, this situation is more complex as we have to 



Here, we demonstrate that the process we have intro- 
duced above leads to rather simple and often determin- 
istic dynamics. We focus on games with two players and 
three strategies with cyclic dominance [4TH5Q] . Cyclic 
dominance among three strategies corresponds to Rock- 
Paper-Scissors games, where each strategy can be beaten 
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FIG. 3: Unconditional (red) and conditional (black) aver- 
age fixation times (measured in elementary time steps) as a 
function of the intensity of selection for a coordination game 
(top) and a game in which A dominates (bottom) . The payoff 
matrices of the games are given in the figures, the population 
size is = 10. Top: In a coordination game (a > c, b < d), 
the conditional and unconditional fixation times, ti{l3) and 
tf{l3) converge to N — i for /3 ^ oo if initially more than 
i* = 117/22 ^ 5.32 individuals play A, compare Eq. (21). 
The lines show the initials states i = 6 (dotted lines) and 
i = 9 (full lines). Bottom: When A dominates B (a > c, 
b > d), the unconditional fixation time ti(/3) first increases 
with /3. For any initial condition z, ti{/3) and (/3) converge 
to N — i in the limit of strong selection. The initials states 
are i = 1 (dotted lines) and i = 9 (straight lines). 



by another one: Rock crushes Scissors, Scissors cut Paper 
and Paper wraps rock. In general, the payoff for winning 
does not have to be equal to the payoff for losing, which 
leads to non-zero-sum games. For simplicity, we set the 
payoff for a tie to zero. Setting the winners payoff to one 
and the losers payoff to —s < 0, the 3x3 payoff matrix 
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S 
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1 


P 




1 





—s 


S 


\ 


—s 


1 






(23) 



For infinite populations, the state of the system is de- 
fined by the frequencies of the three strategies, xr^ xp, 
and xs- Thus, the state space is the simplex 5*3 C M^, 
an equilateral triangle between the three states all R, all 
P, and all S. Apart from the three trivial equilibria, 
the replicator dynamics has an interior equilibrium at 
{xR^xp^xsY — (|, |, which follows from the symme- 
try of the system. The quantity s determines wether the 
interior equilibrium is asymptotically stable (the system 
spirals inwards towards the interior fixed point for s < 1) 
or unstable (the system spirals out toward a heteroclinic 
cycle along the boundaries for 5 > 1). In the zero-sum 
game with s = 1, the system oscillates around the inte- 
rior equilibrium with the Hamiltonian —xrXpXs being a 
constant of motion [HIS]. In finite, well-mixed popula- 







FIG. 4: The discrete simplex of the Rock-Paper-Scissors 
game for the different ranges of s. The three strategies are 
arranged in such a way that cyclic dominance is counter clock- 
wise. For A/" = 33 and s — 1 (top left), the three lines of equal 
payoffs defined by Eqs. ( |34| ) are parallel to the boundaries of 
the simplex. Changing the population size to = 11 (top 
right) does not change these lines, but only the state space of 
the system indicated by dots. For example, the system can 
no longer access the center of the simplex. With decreasing 
s, the three lines of equal payoffs are rotated clockwise (bot- 
tom left, s = 0.5). With increasing s, these lines are rotated 
counterclockwise (bottom right, s — 2.5). Only in special 
cases, the three lines of equal payoffs defined by Eqs. (34) 
intersect with the possible states of the system. The arrows 
in the upper left figure indicate the direction of selection as 
it is induced by the cyclic dominance of the three strategies. 
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tions, the state space is only a subset of (A^ + l)(A^ + 2)/2 
states within the simplex ^'3, cf. FIG. [4j Moreover, the 
dynamics is typically stochastic. Properties such as the 
average drift for a Moran process or the average time 
to reach the absorbing the boundaries are attainable for 
weak selection [46l |49] . To analyze the strong selection 
limit, we adopt the evolutionary process with selection at 
birth and death discussed above for 2 x 2 games. Starting 
with the payoffs from ( 23 ) in a well-mixed population the 
average payoffs read 



: is - 

in 
ip 



IpS, 

- iss, 

- ins, 



(24) 
(25) 
(26) 



where in, ip, and is = N — ip — ip are the number of 
individuals playing Rock, Paper, or Scissors in a popula- 
tion of size respectively. Individuals are selected pro- 
portional to fitness at birth and proportional to inverse 
fitness at death, both with the intensity of selection (3. 
The fitness of strategy X is given by fx = exp[+/37rx]. 
The dynamics on the discrete finite set of states is gov- 
erned by six transition probabilities in each state. The 
transition probabilities in each state {ip^ip, is) change 
to one of the six neighboring states are thus given by 



TY^x{iR,ip,is) 




(27) 



death 



where X, F, and Z stand for P, or S. Note that 
the probability to stay in the given state is given by 
TR^R{iR,ip,is) + Tp^p{ip,ip,is) + Ts^s{iR,ip,is)' 
For strong selection, /3 ^ 00, the system moves from 
each state into one direction with probability one unless 
two payoffs are identical. In the following we address how 
this direction depends on the payoffs in the state and on 
the paprameter s. 

Let us first assume that for given state (ip^ip^is)^ 
we have the unique ordering of the average payoffs from 
(24)-(26). Let tti denote the largest and tts denote the 
lowest value, i. e. tti > 7T2 > tts. The number of individ- 
uals playing the according strategies can be denoted as 
zi, ^2, and is. For Ts^i, we have 



iie^^i 



-/37V3 



ii + i2e-^'^^^-^'^^ + i^e-f^iTTi-TTs) 
X '1 



(28) 



(29) 



where tti — 7r2 > 0, tti — tts > 0, as well as 7r2 — tts > 0. 
For /3 ^ 00, this leads to 



lim T3_ 



1. 



(30) 



All the other transition probabilities vanish. In each re- 
productive event, an individual with the largest payoff re- 
places an individual with the smallest payoff. This holds 
for any unique ordering of the three payoffs. 

If the payoffs are not in unique order, that is, if two or 
more payoffs are equal, at least two probabilities become 
non-trivial. This yields the following three scenarios: 

(i) For TTi > 7r2 = TTs, the individual with the highest 
average payoff is certainly selected at birth. But se- 
lection at death will remove an individual carrying 
one of the two remaining strategies with probability 
given by their abundance. We find 



hm T2^i = 

^^00 Z2 + Is 



(31a) 



^3 



(31b) 



and 

lim Ts^i = . 

/3^oo i2 + is 

Obviously, we have T2^i + Ts^i = 1. 

(ii) For TTi = 7T2 > TTs, the individual with the lowest 
payoff is selected for death with certainty, but se- 
lection at birth is still probabilistic. It is easy to 
see that 



lim Ts^i = . . , 

/3^oo li + l2 

and 
lim Ts^2 = 

/3^oo Zi + l2 



(32a) 
(32b) 



Thus, T3^1+T3^2 = l. 



(iii) For tti =712 = tts, selection is stochastic at birth 
and at death. In this case, we find 



hm Tx^Y = 
As expected, we have y Tx^y = 1- 



(33) 



These results are valid if two or more payoffs are equal at 
a given lattice site, which is only obvious for 5 = 1 and 
might not occur for any lattice site at all in the general 
case of 5 7^ 1. 



With (24)-(26), we can compute the point sets in the 



simplex where two or all three average payoffs are equal, 
depending on the value of s. For np = np = tts this is 
only the center of the simplex, independent of s. For two 
payoffs being equal, we obtain the three linear equations 

TTp = TTS at ip{s ^2) = N - ip{l - s), 

7Ts = 7rp at ip{s-l) = Ns-ip{1^2s), (34) 

TTp = TTp at ip{2s^l) = N{l^s) -ip{2^s). 

However, it is not obvious at which of the discrete states 
iip^ip^is) we can observe equal average payoffs of two 
strategies if the losers payoff is not equal to —1, and 
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it changes direction. Near the corners and on the edges 
the system fixates deterministicahy, but in a central area 
the process spirals inwards if it does not hit the bound- 
ary of the system. However, it turns out that there is 
a largest limit cycle (LLC) depending on s and N and 
that there can be several other limit cycles (OLC) inside 
the LLC. We demonstrate this finding as it is obtained 
from numerical simulations in FIG. [6j showing one sam- 
ple trajectory that ends as the LLC. Note that the term 
cycle actually refers to a hexagon. 



FIG. 5: The probability that the system fixates at the lower 
left corner, e. g. 0^, in a system with N = 33, and s = 1 
for strong selection /3 ^ oo, depending on the initial state 
{in^ip, is)- As discussed in the main text, fixation is stochas- 
tic. Near the corners and along the edges fixation is deter- 
ministic. In a central area of the simplex, the system spirals 
out in a probabalistic fashion. The closer the initial condition 
is to the center, the closer the probability to get absorbed in 
a given state is to one third (fixation probabilities obtained 
from numerical simulations over 10^ realizations). 



especially if the system size is not a multiple of three, 
compare FIG. [4j 

For simplicity we thus concentrate on the case of TV 
being a multiple of three. In general, the strong selec- 
tion behavior of the system is determined by the transi- 
tion probabilities near the lines of equal average payoffs. 
When the population size is a multiple of three, the 
dynamics for the three different cases of s = 1, 5 < 1, or 
5 > 1, is as follows: 

For 5 = 1, we have ttr = tts dit ip = tts = ttp 

at iR = and ttr = ttr at is = N/3. Hence, there 

is always stochastic movement induced by Eqs. ( [31] ) and 
(32) as well as (33). Apart from these points, the di- 



rection of selection is indeed deterministic, which means 
that an individual with higher payoff always replaces an 
individual with a lower one. In certain regimes, near 
the corners of the simplex and along the edges the initial 
condition determines the final state where the system fix- 
ates. In a much larger area, however, the system fixates 
stochastically. In FIG. [5] we illustrate this by showing 
one fixation probability obtained from numerical simula- 
tions of the birth-death process. Due to the symmetry 
of the system, this fixation probability can be either 0^, 
0^, or (j)^ . 

For 5 < 1, the lines where payoffs are equal rotate 
clockwise in our setup of cyclic dominance. In general, 
no states of the finite population system coincide with 
the lines of equal payoffs (except for special cases), see 
FIG. [4j But as soon as the process crosses these lines. 







. - 1 • !/■ 1 • 1 • 1 • \ 
• I • l/l • I • I • 1. • l\ 




1 -^I • 1 • 1 • 1 • [_• l\ 
• 1 / • 1 • 1 • 1 • 1 • 1 • 1 • \ 




1- 1 • 1 • 1 • 1 • 1 • • 1 ■■ 1 • • \ 

a^.V.■^^^^>.•>^ 



FIG. 6: For a population size of A/" = 33 and s — 0.5, the dy- 
namics is such, that fixation does not occur in a central area, 
buy it is deterministic near the corners. For instance, any ini- 
tial condition in the dark blue area will lead to the pure state 
at the lower left. The white area marks the sites from which, 
chosen as initial condition, the system does not fixate, but ap- 
proaches a closed cycle. We show an example trajectory of the 
system that ends on the largest of these cycles (red hexagon) 
starting from the initial state {iR,ip,is) = (1,9,23). Inside 
this largest cycle, there are other cycles such that every site 
except the center is already on such a cycle. 

For s > 1, as the lines of equal payoffs rotate counter 
clockwise (FIG. [4| the system spirals outwards in a de- 
terministic fashion. The movement is no longer deter- 
ministic only near the corners and along the edges but 
everywhere (except for the center), compare FIG.[7| This 
means that if the modulus of the payoff for losing in our 
cyclic game, 5, is larger than the payoff for winning, 1, 
we observe strictly deterministic fixation depending on 
the initial condition. 

In this section, we have shown that a birth-death pro- 
cess with exponential payoff to fitness mapping and se- 
lection at birth and death is able to induce determinis- 
tic movement in the strong selection limit even in 3 x 3 
games. For the transition probabilities Ty^x this limit 
can be performed analytically. It turns out that the mi- 
croscopic dynamics is dependent only on the hierarchy of 
the average payoffs. As for finite systems an analytic 
description of the fixation probabilities (and times) is 
lacking, further examination of this system has to be nu- 
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merical. Under strong selection the patterns that emerge 
show a very interesting regularity; it turns out that apart 
from system size, the results are dependent on the param- 
eter s. 




FIG. 7: Deterministic fixation for a population of size N = 33 
and different s > 1. The dynamics is deterministic every- 
where, except in the center (white dot), where all three pay- 
offs are the same. Depending on the initial state, the system 
fixates to a given pure state. This is indicated by the three 
different colors at each state. For instance, when the system 
is initially at a site in dark blue, the system will fixate to the 
pure state at the lower left with probability one. We show 
one example trajectory (orange arrow) starting from the ini- 
tial state {iR,ip,is) = (10,13,10). By increasing s, we can 
vary the state that is ultimately reached by the system. 



VI. DISCUSSION 

The standard approaches to evolutionary game dy- 
namics such as the Moran process or pairwise compar- 
ison based on the Fermi rule lead to stochastic dynamics 
in finite populations [12]. Even if the direction of se- 
lection becomes deterministic, the time scale typically 
remains stochastic and leads to a distribution of the av- 
erage fixation or mean exit times [29]. Moreover, these 
standard approaches do not lead to a deterministic direc- 
tion of selection in games with more than two different 
strategies [51]. Here, we have introduced a process with 
selection at birth and at death. This process allows to 
interpolate between weak selection, usually considered in 
evolutionary biology, and arbitrary strong selection, such 
that in the extreme case the worst performing individ- 
ual is always replaced by a copy of the best performing 
individual. This kind of selection is sometimes used in 
evolutionary optimization [36l [52]. While the limiting 
case itself may not be of most interest for real biological 
or social systems, which are always subject to stochastic 
noise, we discussed the most important features of this 
limit. In particular, it reveals speed limits of evolution- 
ary dynamics in 2 x 2 games that stochastic dynamics 
cannot cross and shows that in games with more than 
two strategies, the limiting deterministic dynamics can 
have a crucial dependence on the initial conditions. 
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